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Some suggestions and ideas for initiating a statewide 
eaucationai assessment program are provided. Guiding principles for 
an assessment should bmt <1) to specify and define educational goals 
in terms of measurable outcomesi (2) to involve various publias 
extensively^ (3) to use measurament instruments having face and 
content validity; (4) to inelude noncognitive student behaviors; (5) 
to present the results in a form understandable by those outside the 
prof eBSional education community; and (6) to view assessment not as 
an end but as a means of providing useful information to decision 
makers. The objectives of the initial statewide assessment program 
should bmi (1) to collect student performance data that can provide a 
status report on the quality of education in those goal areas 
identified as having high priority; (2) to introduce the concept of 
assessment and its usefulness as a source of information for both 
decision makers and concerned parents or taxpayers ; (3) to provide a 
starting point whereby those managing the statewide effort may gain 
useful experience in operating the programi and (^) to develop a 
method of data analysis that can illustrate the variability of 
performance due to individual differences among students and to the 
social context in which they live* Issentlal components of the 
assessment should include the provision for an advisory committee and 
for selecting its members, conducting meetings to inform citizens and 
school personnel of the nature, scope ^ and methodology of the 
program, data collection materials, data analysis procedures, and 
reporting strategies* (Author/DB) 
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INTRODUCTION 



Traditionally the method of assesslni the quality of educational programs 
and services, a long standing intereat of state government, has been to examine 
such things as the quality of buildings and facllltias, the credentials of 
professional personnel and per pupil expenditures. 

The passaie of the Elementary and Secondary Education Act of 1965 with 
Its requirement that schoola objectively aasess the effects on student achieve- 
ment produced by federally funded programs for the educationally deprived, 
focused attention on measuring the performance of students to assess the effec- 
tiveness of the schools. 

The essential merit of this approach has become increasingly evident to 
educational decision makefs at the state level, and laws mandating statewide 
assessment of the quality of education have been passed in many states. 

Due to tiine constraints. Inadequate budgets, and the speed with which 
many assessmants have been mandated, state departments of educaclon often find 
they lack a workable plan for assessment or the personnel to conduct It. 



_ASSESS>mNT PRINCIPLES 

Experience with statewide assessment programs has led us to the formula- 
tion of the following principles which are dcislgned as a guide for state 
department personnel and others to assist them in optimizing the chances for 
a successful assessment. 

Involve the Congnuntty . Effective educational assessment demands the 
recognition and Involvement of the entire conmunltyi legislators ^ educators, 
parents, students, business managers, labor leaders and other concerned groups 

One method of involving them is to have representatives from each group 
assist in determining what the goals for education ought to be. Since each 
group may have different priorities this could be a time consuming activity. 
The time will be well spent, however, since in addition to determining the 
goals, the participants should also become aware of the needs and constraints 
of the others* For example^ the legislator wants to know about how much pupil 
learning and development the money he appropiates for education is buying. 
He also must answer to his constituents who may not reelect him if they feel 
he is not concerned about the quality of the education their children are 
getting. Since the goals for education most directly concern the students, 
thay should have represent ation in deciding what those goals ought to be. 
Parents want assurance that their children are receiving the kind of education 
that will enable them to cope with the ever increasing compleKity of the world 
In which they live* 

Teachers also have an interest in assessment. Some may have negative 
attitudes because they feel they personally will be evaluated. In addition to 
the valuable contribution they can make, they will be less apt to feel 
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threatened becauae they have been given the opportunity to participate in 
the developmental phases of the program. 

The early involvement of the various Interest gryups should facilitate 
understanding and cooperation when the assessment is conducted* 

Specify and Define Goals , After the goals have been determined, they 
must be defined operationally and behavlorally so they can be measured. 
The conmunity should continue to be consulted in this phase | especially the 
educators. 

An exainple of this type of definition is the goal "To appreciate human 
endeavor in the arty." One aspect of this goal would be to appreciate music. 
An appreciation of music could be defined in bthavioral terms as the nmnber 
of times tapes and records are used* This definition coTresponds to the 
receiving and responding levels of the affective domain (Krathwohlj at al, 
1964). The behavioral objective could then be measured by a frequency count 
of the tapes and records used in the library and those taken out for off campus 
listening. The number of usages and the proportion of students involved 
would be an indicator of the student body's appreciation of music. 

Measuring devices must have face and content validity » The instruments 
should contain an adequate sampling of the specified universe of content* In 
addition J they should be face valid. That is 5 the layman must be able to look 
at the tests and see the relationships between them and the goals being mes^ured. 
If the objective is to measure understanding and the instriment contains items 
that are purely factual in contents the instrmnent would not have content 
validity although it might appear to be face valid. Adequate assessment devices 
must present both. 

Take noncognltive effects of school into accoimt. Society is delegating 
more and more responsibility to the schools for developing learning outcomes 
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which are not skills centered. The appreciation of music goal mentioned 
earlier is one example. Another Is the development of a positive self-concept , 
Although these noncognltlve areas are admittedly more difficult to measurej 
in an aeseBsment program they must not be Ignored in the early phase or they 
most likely will continue to be neglected as the program is enlarged. 

Data presentations should be designed for lay understanding . Possibly 
the most crucial aspect of a successful assessment program is the reporting of 
results. The reports should be in terms that are understandable to the layman. 
Interpretation of statistical data^ particularly that which requires qualifl^ 
cation^ such as test scores , is most effective when Interaction between. the 
receiver and the presenter is possible. However ^ there is likely to be little 
interaction if the results are reported in sophisticated technical terms. 
Four possible alternatives for use in the presentation of data arei expectancy 
tables based on previous yearns performance | comparison with state norms; 
percentage of response to each option of key items | and description of the 
distribution of student scores in terms of the kinds of problems they are 
solving successfully and the kinds which are presenting difficulty. 

Assessment must not be an end in itself . The last principle, which perhaps 
should have been firsts is that assessment must be clearly identified as one 
component of the total education process. Evaluative data are collected to 
meet specific needs and if the data are not related to these purposes they are 
useless. Assessment must provide feedback to enable decision makers at various 
levels to make program modifications necessary for educational Improvement. 
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A MODEL FOR BEGINNING STATE ASSESSMENT 

The politica of assessinent frequently liinit the number of niethods available 
to achieve the principles which were set forth In the preceding section. As a 
result^ state department personnel may have to work under any or all of the 
following constraints. 

Firsts time schedules especially when limited by legislative action — 
most often do not allow an adequate and thorough developinent of assessment 
procedures* 

Second, the resources made available are usually far less than required. 
Thus 5 one must expect that compromises will be made in the operations of the 
program. 

Third, the unavailability of adequate professional staff further complicates 
effective implementation of assessment activities. 

Fourth, the conceptions of assessment held by the several publici who are 
concerned with it are frequently ambiguous and overly optimistic. 

Within the context of the constraints Identified a simplified model with 
limited objectlvas can be implemented. The first objective is to collect data 
which will provide a status report on education within the state in specified 
areas of greatest interest. This model is designed to provide statewide data 
not individual or school building ' data. The most commonly specified areas are 
reading and math because these basic skills are fundamental to most educational 
activities. It Is recommended, however, that even the first level model Include 
data collection In at least one noncognitive area. This recommendation is m.ade 
^because of the human tendency to concentrate efforts upon the areas being evalu" 
ated. Therefore, the failure to evaluate noncognitive areas has the effect of 
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focusing the educational process on the skill development segment of educa- 
tion to the neglect of the equally important but mora difficult to measure non- 
cognitive areas. 

One such area related to the two major cognitive areas, is student 
attitude toward school or learning. This attitude has apparent value to those 
concerned with the educational process and seems a reasonable area of interest 
in which to begin initial efforts in noncognitive data collection. 

The second objective of this first level model is to Introduce operational 
concepts of assessment to the intarested parties. These include school person- 
nel, both administrative and teaching, state government personnel, Including 
legislators and executive office staff, and concerned community groups. The 
consideration of methods of statewide educational data collection by these 
groups will provide them with insight into the limited nature of the kind of 
Information available from a basic model and the problems related to obtaining 
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The third objective Is to provide a plan which will enable state education 
personnel to gain experience In dealing with assessment problems. Among these 
problems are; selecting and securing data collaction devices | negotiating the 
needs for assessment data presented by a heterogeneous public | developing 
communication strategies which minimize destructive conflict and optimize data 
utilization^ and establishing the organizational structura to carry out assess- 
ment programs. 

The fourth objective is to provide a method of analyzing the data to Illus- 
trate the variability of performance due to the Individual differences among 
students and to the social context In which they live. The reason for using 
this method of analysis is to clarify ambiguous perceptions of the interested 
. publics concerning these correlates of performance. Most people recognize that 
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there are individual differences and that these differencei contribute to the 

difficulty of learning. On the ofher hand much of the public appears to ex- 
pect that some simple method of instruction, if properly applied, will over- 
come sunh difficulties. The data produced by this modal can and should be 
used to analyze contextual learning difficulties and differences, thereby re- 
quiring a major consideration of them. 
Components 

The componenti of the first level assessment model include introductory 
activities, data collection materials, analysis procedures, and reporting 
strategies , 

Introductory activities are informative and communicative in purpose. An 
integral part of these activities la the selection of an Advisory Committee to 
consider the objectives of the asseisment and plans for Its accomplishment. 

The selection of the Advisory Committee should be given very careful 
thought. It should be cross-sectional in nature, representing the several 
publics who are concerned with assessment. An additional qualification for 
members should be sincere Interest in education as a community responsibility. 
It is probably not possible under the conditions for which this model was 
designed, to convene the Advisory Committee for the several sessions which 
would result in optimum consldaratlon and support of the assessment program. 
Therefore, the committee function would be that of a review group. Department 
staff, augmented as necessary by independent consultants, should prepare tenta- 
tive objectives and plans for committee review. The committee's recommendations 
should be carefully considered, incorporated If possible, and always given the 
courtesy of a response. 

Following the praparatlon , of initial plans and review by the Advisory 
Committee, a series of regional conferences should be conducted. The conferees 
should represent both the conmunity and professional educators. Special 
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emphaals should be placed upon the effeets of aseessment activities in the 
aehool, and open dlscusaion of thii matter should be given iubstantlal time en 
the conference agenda. 

The arrangements for each conference should include provision for small 
group aessions to facilitate an interchange of ideas among the representatives 
of the different groups. The care with which the Adviiory Coiralttee was 
selected will be reflected in the success of these conferences because, to 
the degree that the participants feel they were represented in the earlier 
planning, thay will be inclined to respond favorably to the pj^rfs! 

The success of these Introductory aetivltiea also deSendB on the content 
of the plans presented. The components which delineate the desired content 
described next. 

The major component is the measuremenC package* For the objective of 
this first level model, presently eKiitlng devices should be selected where 
possible* Standardised achievement teats or minor modlficatiotts of such tests 
are appropriate for reading and math. The exercises used by national asiess- 
ment, which are placed In the public domain upon release, are also candidates 
for utilization. For exsmplis reading and math eKercisea are available In 
standardized testii releaied NAEP items and the Delaware and Michigan state 
aspisimtnt programs. The Dalawarei Michigan and Penniylvanla assessment pro- 
grams have also used instruments that measure attitude toward learning. 

In order to clearly coraaunieate what will ba accomplished with the 
measurement package p it la advisable to provide a content rafarence which 
win ahow how leoroi may be raported in spteifia skill or bahavlsr ralated 
fom, Co^tant raforance, ai uaed hero, maani an iKamplo or dticriptlon of 
ehi bihivief Itnplled by the rsBponie to tha quiicion. For iMnplt, the ateps 
to provide §u6h a rtferinee for a iample rsadlng comprthtnalon Ittm baaad 

o 

ERiC**^** parairapl)! afo as follows t 
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.Identify a aerias of akllls ttited by the apeeiflc questlone asked, 
eueh as making InfeMnGes, detecting mood, or recognlilng factual 
detail. 

, Prepare or eeleet an equivalent paragraph to that preiented In the 
test and show how it will be used to Illustrate the obtained reaulte, 
•After testing, the percentage of students who auccesafully respond to 
each kind of question, indicating accompliihment of the skill of Interest 
should be reported. 
In the case of attltudlnal questlone, deacrlptloni of specific approach 
or avoidance behaviors, evoked by situations or persons, are the content 
reference* A Llkert type response format to this sort of Item will provide 
information about the proportion of students who respond In each direction 
(approach or avoid) and also about the intensity of the responses to the 
behavior in question. After testing, If it is decided not to report the 
actual questions, the underlying constructs can be described and the response 
frequencies can be related to them. 

The final Ingredient of the measurement component for this model Is the 
collection of relevant educational content Indlcatorrj which may be used to 
classify the school score distributions and thereby provide information useful 
in generating hypotheses about the anteGedents of student performance* 
Examplee of these indicators are socio-economic status and teacher verbal 
ability (see Coleman et al). 
Data Collection 

Preparing for the actual collection of data for this model requlrei a 
decision about the reference population . It may be a population of schools 
or Individual students* If the school is the unit tgt analysis it la fre-=- 
quently easy to iecure a sampling frame because inost state departments maintain 
^ « list of operating SQhools In thiir states. The problem of obtaining 
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adequate representatlQii within the sehoola however, is not so easily iolved, 
Assiming a high degree of variability within a sehool, rather large student 
samples are required to provide a sehool score with a reasonable degree of 
prealsion. Therefore, it is reconmended that for this model a sample of 
, randomly selected sehools be used to generate the state educational status 
data specified in the first objectivei An example of this procedure follows. 

Suppose that there are 3^000 elemeatary schools in the state and that the 
measure of interest has a standard deviation of fifteen. In this situation, a 
sample of 200 elimentary eehools will provide an estimate of the state average 
which will deviate from the actual average no more than two score points with 
95% confidence. If information is available which Is known to relate to the 
student characteristic of Interest* stratification of the sample will result 
in greater precision! and a reduction of sample siie becomes possible (see Klsh) 

It is possible I however, to select a sample which allows analysis of 
individual student data although the complaxlty and therefore the risk of 
administration error increasae. 

The design requires that the probability of selecting any one student 
remain equal even though we do not know the namai of all itudenti in the state* 
M eKample of this procedure !§ furnished in AppendlK A, 

The school sample described above does not permit the analysis of any 
individual student output data unless some generally untenable aisianptions 
are made about the random assignment of students to schools. Neither does 
such a sample pemlt estimates to be made for Individual school districts* 
Because of the anticipated substantial variance among schools within districts , 
it would probaby be ntcessary to sample most or all of the schools in each 
district in order to obtain suitable district estimates, Aa In the case of 
data collection at the state level > the model at the district levil would not 
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necessarily require the tasting sf all students within the selected schools. 
However, such all-lneluslve data collection might be more feasible than 
within-school sampling of Individuals. 

District estimates would, of course, enable districts to compare them- 
selves to other districts "like them" as defined by a wide variety of situa- 
tional variables. Reports could be generated to facilitate these comparisons 
in terms of both district averages and district score distributions."'" 
Analysis Procedures 

The objective of this first level model la to provide a description of 
the position of the schools in the state with reference to the educational 
objectives deemed Important In that state. The data may therefore be analyzed 
according to straightforward daserlptlve Information. Frequency distributions 
for the state, the measure of central tendency, probably expressed as a mean, 
and a measure of variability, such as the standard deviation can readily be 
produced. The second objective of the model however, is to direct attenclon 
toward the different levels o£ achievement of student groups as a way of 
highlighting both differential difficulty of learning and the areas in which 
hypotheils generation ml|ht be productive. 

For these purposes, analysis of the data should ninimally Include 
dlitributiona classified by the qualitative Information colleeted to reflect 
the educational context. This Information does not have to be quantifiable, 
although the idea of levels may be appropriate, In order to be useful In the 
analysis. A minimum four cell claisification of distributloni is recoramanded, 
80 that possible interactions may be detected. For example, the data might 
look like thisi 
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These district asaeietnent ideas were suggested by Donald Trlameni 
EduQational Studies. ' 



Attitude Toward School 



High Staff Training 



Low Staff Training 



High SES 



47.8 
49.5 
46.2 
52.1 
51.0 
51.7 
54.7 



44.6 
47.4' 
42.6 
46.8 



X ^ 50.2 



X s 45.8 



Low SES 



50.4 
47.8 
51.2 
48.8 



50.2 
47.3 
52.0 
47.6 
54.5 
46.8 
44.3 



X 



= 49.5 



X 



s 48.6 



An inspection of these data suggests that, in terms of attitude toward 
school J teacher training has a more significant association than socio-econoinlc 
leve.l. There are appropriate statistical tachnlques for detenninlng the 
probability that the observed differences are actual rather than due to chance. 
Two possible procedures are two-way factorial analysis o£ variance or a 
Friedman two-way analysis for ranked data. 

The analyils described here la useful in Identifying interrelationships 
which should be examined further. The purpoie of this additional examination 
Is to discover what experiences chlldrin have which may be modified to pro- 
duce desired chanies in output, in this case Improving attitude toward 
learning. Observation of schools located In high scoring and low scoring 
groups in the classification tables, should suggest productive ways of 
changing the learning situation. 

The limitations of this analysis, however, include the possibility that 
less distinct relationships may not be revealed. Also, it becomes eKtremely 
complex to examine the effecti of several conditions taken as a group. 



If the data collected on both studtnt output and tht conditions of 
learning are quantlf iablCp and if then is reason to believe that relation- 
shipi are fairly uniform across the range of scores from high to low, it is 
appropriate to use multiple correlation techniques for the analysis. These 
procedures allow more complex relationships to be considered and provide a 
method for examining the unique contribution of many variables in a syatematic 
way. Partial and semi-partial correlation techniques ara included in this 
classification (see Nunally) * 

The principal use of these correlation ttchniquis, as in the case of the 
two-way classification analysii, is to identify variables which should be 
studied for poisible influence upon the experiences students have, and therefore 
upon what they learn. Results obtained from these techniques do not suggest 
corrective action directly, but are the first part of a two phase process of 
educational change. The second phase requires alteration of learning conditions 
or of the arrangement of learning eKperlences which can then be evaluated by 
a eubsequent assessment. 
Reporting Results 

Interpretation of statistical data, particularly that which requires 
qualification^ such as test scores, is moit effective in a context in which 
interaction between the receiver and the presenter Is possible^ Therefore, the 
ideal method of interpretation includes a perional interface between the 
concerned school personnel and a presenter who knows both the nature of the 
data and the method of analysis. If the whole process is to be cost effectlys, 
a discussion of implicatloni consistent with the results and suggeitlons of 
alternative courses of action must be included* 

In repDrtlng the results of the itate itstus asaessment to legislators ^ 
state boardi of education^ the governor 'e staff and other decliion makeri, 
the pirsonal Interface is extremely Important* If itandardized tests or NAEP 
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exercises and proceduris have been used, the results can be compared with 
regional and national data. 

If dlstrlet or school building data have bean collectad, more detailed 
methods of reporting results are needed. Several options for reporting 
results are provided In Appendix B, 
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IMTER-DISTRICT CO^ARISONS 

In the foregoing discussion the primary focus Is on an aisessment program 
that views the state in Iti entirety (I.e. the sample Is drawn from the state 
population for the grade levelCs) of Interast) . Coniiderable Interest Is being 
directed toward the use of student ptrfonnancc data gathered In an assesiment 
to compare school districts. 

The following la a brief discussion of two considerations that should be 
taken Into account when inter-dlstrlct comparisons are to be made. One relates 
to the sample design and the data analysis. The other pertains to soclo-eeonomie 
and other condition varlablas - both situational and individual - and their 
critlcallty when Inter-dlstrict coni> rlsons are to be made. 

Sample Design and Data A,i«1^M^, As was Indicated earlier some generally 
untenable assumptions must be made if district estimates are made from a sample 
of students where the population frame la the state in its entirety. Should 
inter-district comparisons be desired, the population from which schools or 
students are drawn should be the individual district. Since most school districts 
either maintain master lists of students or can obtain a list with minimum 
difficulty, the less cOmpleK method of drawing a sample Is to use a simple random 
sample of students. It is possible, but more difficult, however, to use a two 
step cluster sample technique in some districts (i.e. in step 1 select a sample 
of schools I m' step 2 draw a sample of students from each of the previously 
selected schools), while using a iimple random sample procedure in others. 
This combined approach might- be feasible when exceptionally large school districts 
do not maintain or have available student lists. In large districts the two 
step cluster sample technique might be used while In smaller districts where 
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student lists are available, a simple random sample of students could be 
drawn. The major difficulties In using the two step cluster sample technique 
relate to bias in local scores caused by the possibility of greater student 
homogeneity (I.e. less variability) in an individual school. 

Most states have a wide range in the nu^er of students enrolled in their 
school districts. When drawing a student sample, larger districts may be col- 
lecting data on, for example, 102 of the population, while in smaller districts 
it may be necessary to gather data from the entira population. When aggregating 
district data upon which to make inferences statewide, arithmetic weighting of 
district data is used when computlni estimates of parameters and their standard 
errors. The disproportionate allocation of students in individual districts is 
eoffipensated for by using inverse weights in the statistics. That is, in the 
larger district where 10% of the student population is sampled and therefore 
under-represented, the data are weighted up, while in a smaller district where 
data are gathered on all of the students and therefore over-represented, the data 

are Weighted down. The correct weight for a district can be determined by Ni 

N 

where NJ is the number of students in the population from the Jth district and 
N is the nmnber of students in the total population. 

Crltlealltv of Condition Variables . When district comparisons are desired, 
special "attention should be directed toward those conditions of learning that 
may be associated with student performance. 

Student background characteristics such as soclo-eoonomio status, attitudes 
and aspirations have been found to be associated with (not to be confused with 
caused by) student achievement. Furthermore, other school variables such as 
the quality of the instructional staff (e.g. staff training) and the availa- 
bility of financial resources also have shown an aisoeiatlon with student 
achievement . 

rn^^^^®®®^"^^--'-®* become critical when inter-district comparisons are to 
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be made. For example, cdmparing an inner^city school district serving children 
from economically deprived families with the affluent suburbs surrounding could 
be grossly misleading. The inner^city school district , when the environment 
of its students is considered, might be making a more substantial contribution 
to student performance than its tnore affluent neighboring school districts ^ 
when the environment of their students is taken into consideration. Other 
examples could be cited j however ^ the important point is that as nearly as is 
possible, school districts should be compared with those that have similar 
characteristics that would be difficult to change by educattTOal policy 
decisions (e,g, socio-^economic status). 

In comparing similar districts, it is important that categorization of 
districts reflect those difficult to change variables that are associated with 
differences in output* Although the ultimate objective of assessment is to 
provide information which will enable decision makers- to Improve the educational 
performance of all children^ it is naive to eKpect that such Improvement will 
result Immediately, Therefore, the condition variables should be considered 
in designing alternative school programs which show promise of improving 
student performance (e,g. greater utilization of prior student experiences), 
Statlitlcal procedures for determining categories include expectancy tables ^ 
regression analysis, analysis of vafiance and analysis of covarlance. 
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This plan provides some, suggastlons and ideas for initiating a statewida 
educational asseasinent program. 

Guiding principles for an assessment should be: 1) to specify and 
define educational goals in terms of measurable outcomes; 2) to involve various 
publics eKtensivelyi 3) to use measurement instruments having face and content 
validity; 4) to include noncognitive student behaviors; 5) to present the 
results in a form understandabla by those outside the professional education 
coranunity; and finally 6) to view assessment not as an end unto Itself but 
as a means of providing useful information to decision makers* 

The objectives of the initial statewide assessment program should be: 

1) to collect student performance data that can provide a status report on the 
quality of education in those goal areas identlfiad as having high priority; 

2) to introduce the concept of assessment and its usefulness as a source of 
information for both decision makers and concerned parents or taxpayers ; 

3) to provide a starting point whereby those managing the statewide effort may 
gain useful experience in operating the prograrai 4) to develop a method of 
data analysis that can illustrate the variability of performance due to indivi- 
dual differences among students and to the social context in which they live* 

The eesential eomponents of the initial statewide assessment should ' 
include such Introductory activities as the provision for an advisory conunit- 
tea and for selecting its members, conducting meetings to inform Interested 
citizens and school personnel of the nature j scope and methodology of the 
assessment, program. Other essential componenti are data collection materials, 
data analysis procadurea and reporting strategies , 
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In ordar to make Inter-dlstrict comparlsoni , the sample must be repre- 
sentative of the individual districts rather than the entire state. Simple 
random sampling of itudenta in the districts is recotnmended when feasible. 
The more complex two step cluster sample technique may be used in large dis- 
tricts as an alternative. 

When comparisons are made, the condition variables must be considered in 
categorizing the districts or the comparisoni will not have meaning. Expectancy 
tables J regression analysis^ analysis of variance and analysis of covarlance 
are statistical procedures thay may be uied to determine categories. 
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APPENDIX A 

Sample selection - individual student sampling 
unit - equal probability 

Arrange the schools in any order, with the number of students in the 
grade of interest specified for each school. 

Assign sequential blocks of numbers to each school. The size of the 
block is determined by the number of students In the target grade. 
Inflate the block size by a specified percentage to allow for increasei 
in enrollment. This percentage should be taken into account when 
determining sample siize so that the desired sample size will reinain 
after the shrlnkaga caused by empty sampling frames (i.e. numbers in ' 
the block not having a cdrresponding Individual). 

Using a suitable random selection procedure, (table of random numbers, 
computer program, etc.) determine the numbers which are to be included 
in the sample. 

Assign selected numbers to each school block, converting them to the 
saquentlal number within the block (eg. in the school with block 
2998-3092, 2996 becomes 1, 2999 bacomes 2, 3024 becomes 27.) 

Instruct school coordinators to arrange class lists in any order and 
assign the numbers 1 through N to the total group of students (not 
recyclini through classes.) 

After the asaignmenti have been made, thg Instructions are to open the 
envelope containing the randomly selected numbers and administer the 
test to those students whose names on the school list correspond to 
these numbers. 
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OPTIONS FOR REPORTING RESULTS 

O ption One "- Interface 
Step 1 

Proeeas data and prepare written reports 
A* Alternative formats 

1* EKpectancy tables baaed on previous year's performance * 

2, Comparleon with state normSi 

3* Pereentage of responses to each option of "key" Items— those 
where content Is face valid and which relate most highly to 
other items in the subscale^ thereby reflecting well th© 
concept Tneasured. 

4. Description of the distribution of children In terms of the 
kinds of problemi they are successfully solving and the kinds 
which are presenting difficulty. If available, report 
response patterns for high scorers, middle scorers, and low 
scorers* 

Step 2 

Assemble and train a corps of data interpreters. 
A. Sources of personnel 

1. State department personnel 

2. College and university itaff, graduate students. Interns, etc, 

3. School personnel 

4. Intarmedlate unit office staff 

No single source can provide enough personnel to comprise the neceasary 
teams I but a combination may make possible iuffielent ntMbers to provide one 
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Interpretive viait to each district. 
B, Training 

1. Sessions concerning test developmenti Item to goal correspondence ^ 
raliability, group statistics versus individual statistlci, and 
presentation of comparative data classified by school variablas. 
Methodology includes presentations using actual state data. 

2, Sesilons exploring implications of results and identifying 
strategies for the development of alternative action plani 
where need is Indicated, These, sessions should include 
specialists in the areas assessed. The objective of these 
sesiloni is to stimulate action through provision of suggested 
approaches or avenue of eKploration based on assessment resulti. 

Step 3 

A. Schedule presentation sessions with each school district. It 

is recommended that a work group comprising admlnlitratlve staffs 
teachers^ parent advisory representatives ^ at leaet one board 
member 5 and other interested people be included. 

Step 4 

Organize a consortium of educators across the state who can serve as 
a resource to the school personnel who will be working on altered approaches 
to learning and peraonal de'velopment for their students. 

The resource requirement for a program of this sort is dependent in 
part upon voluntary professional comnitments of the state's educators. It 

also requires allocation of funds for personnel time on the part of many ' \ 

initltutions In addition to the Department of Education* If properly ap- I 

proacheds there is enough professional conmitment among profeisional educators I 

i 

to achieve some degree of this resource allocation. J 

o , , .. . -.-.J 

ERIC I 
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For a state with 500 districts the requirements o£ the interpretation 
team are approximately as follows. In a one month period the 500 districts 
would be visited on the basis of 1-1/2 days per district. AHowing soma 
surplus for contingencies, 78 persons working 11 days each will be required. 
Based on a per diem expense of $25 plus an average transportation cost of 
ten cents per mile for 150 miles, the total cost will be less than $30,000. 

Such an approach to the problem of reporting and utilizing assessment 
data has the best chance of not only assuring positive follow up but also 
of reducing hostility in both the schools and their communities. 

Option Two - Regional Interface Model 

It is recognized, however, that resource constraints may. not allow so 
comprehensive a conmitment. Therefore, a less optimistic alternative can 
be conceived. 

This model would follow the same basic steps outlined in Option One, 
but rather than providlni individual district data presentations as outlined 
in Step 3, it would utilize a series of regional meetings in which about two 
representatives from each district would participate. The format suggested 
for the regional meetings is the model used for the training sessions de- 
signed for the data interpreters (Step 2 of Option One). The corps of data 
Interpreters In this case would be smaller since 30 districts can be accom- 
modated in each regional meeting requiring only four data interpreters for 
each session. The person days required are thus reduced to about 120. If 
we assume that local districts will pay transportation and housing costs of 
their representatives, and also aisume the professional Investment of the 
state's educators, the cost for this kind of program would be about $6,000. 
It must be recognized that the technical experts used for training data 
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Interpreters will not be available for the 17 regional meetings and that 
their contribution would be filtered through two "student" levels, the data 
interpreter and the school district regional meeting attendees. However, 
the concentrated thinking about the problems and the utiliiation of asiessment 
results that such a program would encourage would still stand a fair chance 
of having an impact on the education in a state* 

Option Three - Lacture^Dlscuision Model 

In order of desirability, a third level of data presentation might be 
to conduct five regional meetings with an average of 200 participants comprise 
Ing two representatives from each of one hundred districts* Real but non-- 
identifiable state data could be presented in a visual Ir.ctura'-dlscusslon 
mode In an auditorium setting with questions limited to a small sub-group 
of school representatives. These interrogators could collect their questions 
from discussion aectloni prior to the question and answer period* Such 
meetings could be handled by State Department personnel augmented by aome 
representation from the corps of experts available within the state. Follow- 
ing these regional meetings the assessment data would be released to each 
local school district — preferably scheduled to allow time for district staff 
consideration before it becomes public. 

Option Four - Mail-out Kits 

The least desirable^ least expensive and not only least useful^ but 
posaibly uaelesSs preientatlon Is a mall-out kit of charts, lists and printed 
discussions which would be sent to the suparintendent in each district. To 
optimize its very slim chance for utility and positive impact, the mail--out 
kit should Include the following components* 

O 
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First, the data should reflect the maximum amount of knowledge on the 
part of the State Department of Education about the school district to which 
it is directed. Second, it ahould be personalized in a form which describes 
what the students are like who represent several points on the distribution 
of data for the school. 

For example, a student in the upper quarter of the score ranie on the 
reading scale could be described in terms of. skill content of the question 
to which he responds correctly. The actual pattern of responses should be 
included. The report could read "Students in the highest quarter of the 
score range tend to answer correctly items which require Inference two out 
of three times, and items which require locating factual detail four out 
of five times. There are 178 (201) students in this range from your school. 
For comparison, students from other schools of your community type (Type 3) 
respond correctly to inference items three out of four times and to factual 
detail questions also three out of four times. Seventeen percent of all 
community Type 3 students score in the highest quarter." 

A similar discussion of middle and low scorers should be provided. 
Backup material describing community type and resource availability should 
also be Included, In addition, any of the written alternate forms of pre- 
sentation suggested in Option One as material for the more desirable personal 
interpretation could be ^provided. This published form of presentation could 
also be used as a supplement to any of the options. 

In summary, the most productive Interpretive format for State Assessment 
data is conceived to be an individually tailored personal presentation to 
school district personnel. The least productive interpretation Is a written 
report. A State Department of Education is urged to locate the resources to 
do personalized data interpretation rather than a less Involved and therefore 
less expensive mail-out, ■ 



